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1. 


Introduction 


I 


I 


The  analysis  of  several  stochastic  models  gives  rise  to  ergodicity  and  steady- 
state  equilibrium  studies  of  Markov  chains.  When  a  Markov  chain  is  irreducible  and 
aperiodic,  with  state  space,  C,  and  transition  probabilities,  {p^},  a  necessary  and 
sufficient  condition  for  ergodicity  is  that  a  solution  of  the  following  system  exists. 


ir. 

J 


y]  p  •  i  ^ ^  —  o  '• v  =  i 

ieC  ieC 


The  study  of  the  above  system  requires  explicit  knowledge  of  all  the  transition 
probabilities,  {p^},  and  for  large  dimensionality  state  spaces,  C,  the  search  for 


its  solution  becomes  practically  impossible.  Recognizing  this  fact,  several  re¬ 
searchers  provided  simplified  ergodicity  conditions,  for  certain  classes  of  Markov 
chains.  Pakes  (19691  derived  a  sufficient  condition  for  ergodicity  of  Markov 


chains,  {Xn),  that  is  solely  based  on  the  expected  conditional  drifts,  E(Xn^1  -Xn  |Xn“k). 


n+1  n 1  n 

Kaplan  (1979)  provided  a  criterion  for  nonergodicity  of  a  Markov  chain,  which  is 
again  based  on  the  expected  conditional  drifts,  and  on  certain  imposed  conditions 
on  the  transition  probabilities.  Szpankowski  (1981)  generalized  the  conditions  given 
by  Pakes  and  Kaplan,  using  Lyapunov  functions.  Szpankowski's  approach  is  especially 
useful,  when  the  Markov  chain  state  space  is  not  the  natural  numbers. 

In  this  paper,  we  generalize  Szpankowski's  sufficient  conditions  for  ergodicity 
of  Markov  chains.  In  addition,  we  provide  conditions  for  steady-state  equilibrium 
of  irreducible,  aperiodic,  and  ergodic  Markov  chains,  with  denumerable  state  spaces. 


2.  Ergodicity  Conditions 

Let  {Xr}  denote  a  generalized  stationary  Markov  chain,  with  denumerable  state 
space,  C.  We  allow  multidimensional  states.  Since  the  state  space,  C,  is  denumerable, 
we  assign  a  unique  natural  number  to  each  state.  Then,  the  expressions  keC  and  X^=k 
denote  respectively,  the  state  in  C  that  has  been  assigned  the  natural  number  k,  and 
the  nth  in  time  (multidimensional)  datum  from  the  chain  being  identical  to  the  state 
identified  as  k.  Let,  {p».  ;£,keC},  be  the  set  of  the  stationary  transition  proba- 


bilities  of  the  chain.  That  is. 


P£k  =  P^Xn+l  :^Xn“k*  ;  keC 

Let  us  denote, 

P£>  &  p(xn=£|xo-k) 

Let  R  denote  the  real  line,  and  let  V:  C  -*■  R  be  a  functional  defined  on  the 
state  space,  C,  such  that, 

3  a  :  V(k)  >_  a  >  -®  ;  V  keC  (1) 

Let  us  then  define, 

S(k)  =  E{v(Xn+1)-V(Xn)  jxn=k}  =  tva)-V(k)]p£k  ;  keC  (2) 

leC 

The  quantity  S(k)  in  (2)  represents  conditional  expected  drift  of  the  Harkov 

chain,  (VCX^)}.  From  now  on,  whenever  the  expected  value,  E{f  (X^)  }  =  ^  f  (£)P(Xn«£.) 

JUC 

is  used,  for  some  functional,  f  :  C  -*■  R,  the  implied  assumptions  will  be  that, 

*  0  ;  V  A  4  0,  and  that  at  least  one  of  the  partial  sums,  23 

ItA  l:fU)<0 

and  23  f (£)P(Xn*£),  is  finite.  The  series,  /]£ (l)P(Xn°.£) ,  is  then  inamblguous, 

l:  f  (£)  >0  UC 

and  when  it  converges,  it  converges  absolutely;  that  is,  f(XR)  is  then  summable.  We 

now  express  two  propositions. 

Proposition  1 

Let  there  exist  t  in  C,  and  some  positive  finite  number,  b,  such  that, 

V(£)  <  »,  P(Xq=£)  =1  and  S(k)  <  b  ;  V  keC 
Then,  for  all  n,  the  expected  values,  E{  |v(Xn+^)-V(Xn)  | }  *  23l^k)|p£^  an<* 


E { J  V ( X^ )  |  }  =  ^  1  V(l)  |pj  ,  are  both  finite.  Thus,  in  conjuction  with  (1),  we 
keC 

then  obtain, 

a  <  E{V(Xn+1)}  =  E{V(Xn)}  +  E{V(Xn+1)-V(Xn)}  =  E{v(Xn)}  +  ^S(k)p^  (3) 

keC 

Proof 

Let,  C"  =  {k  :  S(k)  <  0}  and  C+  =  {k  :  S(k)  >  Oh  Then,  0  <  ^  SOOpJj*  1 

keC* 

i  b  ^p£^  =  b5  tbus,  the  series  ^  S(k)p^  is  then  defined,  for  all  n.  That  is, 
keC  keC+ 

2  S(k)Pk!e)  < 00 ;  w  n  (4> 

keCh 


Let  us  now  turn  to  the  variable  V(Xn) .  Due  to  (1)  and  the  assumptions  in  the 

proposition,  we  have,  |vCd) |  <  Let  us  select  n,  and  let  us  temporarily  assume 

that  V(Xn)  is  summable;  that  is,  ^  |v(k)  |p(Xn“k)  <  «®  ;  VA<I.C.  Then,  in  conjuction 

k£A 

with  (1) ,  we  obtain. 


S(k)  >  a  -  V(k) 


Thus, 

°  i  E  s<t,pk£)  ± a  E  -  E  v<k)p«„-k)  »  (5> 

keC-  keC“  keC" 

So,  if  V(X^)  is  summable,  we  conclude  from  (4)  and  (5), 

E{1V(Xn+l)"V(Xn)f  }  "  2lS(k)IP^  "  -  £  SOOpJJ  +  EsCWpJ?  <  “  (6) 

keC  keC“  keC* 


Now,  since  V(Xn+1>  *  tV(Xn+^)-V(Xn> ]  +  start*-n8  with  the  initial  condition 

|V(£) j  <  °°,  we  can  easily  complete  the  proof  of  the  proposition  by  induction, 
observing  that  the  summability  of  V(X  )  and  V(X  . )-V(X  ),  implies  summability  of 
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Remark  1 

If  in  proposition  1,  the  state  L  is  such  that,  0  <  P (XQ=d)  <  1  and  |v(d)  j  <  08 , 

the  results  in  the  proposition  hold  if,  E{ | V(Xn> | },  E{ |v(Xn+1)-V(Xn>  | },  E{v(Xn>}, 

and  E{v(Xn+^)-V(Xn) }  are  respectively  substituted  by  the  conditional  expectations, 

E{  |  V  ( X  )  j/X  =d},  E{|V(X  )-V(X  )|/X  =d},  E{V(X  )/X  =£},  and  E{V(X  ,.)-V(X  )/X  =£}. 
n  o  n+i  n  o  no  n+1  n  o 

Proposition  2 

Let  there  exist  a  positive  finite  number,  b,  such  that,  S(k)  _<  b;  V  keC.  Then, 


-dim  ^  *  S(k)p 
n"~k£C 


(n) 

k d 


>  0 


V  -dec  :  p(X  =£)  >  0 

O 


(7) 


Proof 

Let  us  assume  that  (7)  is  false.  Then,  there  exists  some  state  -deC,  such  that 

P(X  =-d)  >  0,  and  for  this  state  there  exist  6  >  0  and  natural  number  N,.,  such  that, 
o  o 

£sik)p^  <  -6  ;  V  n  >  (8) 

keC 

From  expression  (3)  in  proposition  1,  modified  as  in  remark  1,  in  conjuction 
with  (8) ,  we  then  conclude. 


E{V(X  |X  =£}  <  E{V(X  ) |X  -£}  -  6  ;  V  n  >  N. 
n+1  o  n  o  —  o 


And  thus. 


EfvtXjj  +k)  |Xo=£}  <  EtvCXjj  )  |XQ-d}  -  k6  ; 

6  6 

But,  from  proposition  1  we  conclude,  E{V(X^  )  jx  ad} 

6 

dim  E{V(X  )  jx  =t }  *  -°°,  which  is  impossible  due  to  (3)  . 
n-*»  n  ° 

is  true. 


V  k  (9) 

<  08 .  Thus,  (9)  gives 
We  thus  conclude  that 


then, 

(7) 


Remark  2 

The  statement  in  proposition  2  holds  for  any  process,  lXn),  with  denumerable 
state  space,  whose  expected  drifts  are  time  invariant,  if  E{|v(X  )|}  <  “,  and  if 


5 


the  probability,  ,  in  the  proposition,  is  replaced  by  the  probability,  P(Xn  =  k) . 


We  now  express  the  main  result  of  this  section  in  a  lemma.  The  lemma  states 
a  sufficient  condition,  for  ergodicity  of  a  generalized  irreducible  and  aperiodic 
Markov  chain. 

Lemma  1 

Let  {X^}  be  a  generalized  irreducible  and  aperiodic  Markov  chain,  with  denumerable 
state  space,  C.  Let  t  be  some  state  in  C,  such  that,  P(Xo=£)  >  0.  Let  there  exist, 
e  >  0,  HjCC,  a  set,  {dk},  of  positive  finite  constants,  and  a  positive  and  finite 
constant  b,  such  that. 


S(k)  <  -  e;  V  keHL  and  -  e  <  S(k)  <  b  ;  V  k d<2  *  C  -  ff 

2^  dk  <  «>  and  p^  <  dfe  ;  V  n,  V  k e«2 
keH2 

Then,  the  Markov  chain,  (Xn),  is  ergodic. 

Proof 

Since  the  Markov  chain  is  irreducible  and  aperiodic,  the  limit,  =  £im  pf^  , 

n-*<® 

always  exists,  and  it  is  independent  of  the  state,  t. 

Now, 


Es<k>pu  - 

E  s<k>fff  ♦ 

E 

s(k)pk£  -  "  e 

t1  -  E  *£’] ♦ 

23 s(k)p 

(n) 

k£ 

keC 

keH 

keH, 

4. 

) 

k  cH2 

keH2 

-  E  s(k)pw 

<  - 

■  e+£3  [StkJ+eJp^ 

(10) 

keC 

k  cH2 

Since  |S(k)+e 

ip^  1  (b+e)dk 

;  v 

k eH^  ;  Vn,  and 

23  dk  <  °°,  £be 

series  on 

the 

keH2 

side  of  (10) 

converges  uniformly 

in  n.  Hence, 
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im  ^sOOpJJ  <  -  e  +  iirn  ^  [SUl+elpj^  =  -  e  +  ^  [S(k)+e]Tik  (11) 


But,  for  the  simultaneous  satisfaction  of  inequality  (11),  and  inequality  (7) 

in  proposition  2,  it  is  necessary  that,  ^  tS(k)+£]iTk  >  0,  which  implies  that  >  0, 

k  cH2 

for  some  k  in  H  .  Thus,  the  chain  is  ergodic. 


The  conditions  in  the  lemma  are  relatively  general,  and  they  imply  the  conditions 
for  ergodicity,  given  by  Szpankowski  (1981).  In  fact,  the  latter  conditions  evolve 
as  a  corollary  of  lemma  1,  which  is  expressed  below. 

Corollary  1 

If  the  set  H..  in  the  lemma  is  finite,  then  d,  =  1  ;  V  keH,.  satisfies  the  condition, 
2  k  ' 

d^  <  °°.  Thus,  the  irreducible  and  aperiodic  Markov  chain,  {X^},  is  then  ergodic. 


3.  Equilibrium  Conditions 

In  this  section,  we  consider  steady-state  equilibrium  conditions,  for  generalized 
irreducible  and  aperiodic  Markov  chains.  We  use  the  same  notation  and  quantities,  as 
in  section  2.  We  first  present  a  proposition. 

Proposition  3 

Let,  (Xn),  be  a  generalized,  irreducible,  and  aperiodic  Markov  chain,  with 
denumerable  state  space,  C.  Let  there  exist  a  positive  and  finite  constant,  b,  such 
that,  S(k)  _<  b  ;  V  kcC,  and  let  the  chain  be  ergodic.  Then, 


0  <  2s(k)\  -  b 


;  where. 


\  ■  pu 


n-+«o 
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Proof 


Let  the  subspaces,  C+  and  C~,  be  defined  as  in  the  proof  of  proposition  1. 

We  then  obtain,  for  given  n, 

ZIm  ^S(k)p^)  =7te(  V  SCkJp^  +  ]ES(k)p£f)  < 
n~°  keC  keC"  keC+  ' 

<  Zim  V'  S(k)p£^P  +  Zim  \  '  S(k)p£^  (13) 

n"~  teC-  n~°  taC* 

Since,  0  £  S(k)  £  b;  V  keC*,  and  since  converges  uniformly  in  n 

k  eC+ 

(see  Chung  (1960),  Th.  4),  the  sum,  ^  ^  S(k)p^\  converges  uniformly  in  n  as  well. 

ket+ 

Thus,  Zim  y  '  S(k)p^^  =  y  '  S(k)lT^.  Since  S(k)  <  0  ;  V  keC-,  applying  Fatou's 


lemma,  we  obtain. 


Zim  SCkJp^  <  S(k)irk  <  0 


From  the  above,  in  conduction  with  (13),  we  thus  obtain. 


0  -  £im  E  S(k)pk?)  -  E  s<k>\  +  E  s<k>\  “  Vs(k)\  <  b 

^  keC  keC"  £(+  tc 

The  proof  of  the  proposition  is  now  complete. 


We  now  express  the  main  result  of  this  section  in  a  lemma ■ 

Lemma  2 

Let  the  generalized  Markov  chain,  {Xn>,  be  irreducible,  aperiodic,  and  ergodic, 
with  denumerable  state  space,  C.  Let  either  one  of  the  following  conditions  hold, 
where  V(*)  is  a  functional  as  in  section  2. 
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(i)  X)>V(k)!\< 

keC 


,  A  „ .  (n) 

;  where  =  4.1m  p^£ 


(ii)  There  exists  positive  and  finite  number,  B,  such  that. 


E{ I V(X  ..)-V(X  ) l/X  =K>  <  B  ;  V  keC 
n+i  n  n  — 


Then, 


£s<k>\  - 0 

keC 


Proof 

(a)  If  condition  (i)  is  satisfied,  then  V(Xq)  and  V(X^)  are  both  integrable  and 

they  have  the  same  distribution,  if  P(Xq  =  k)  =  Thus, 

0  =  E{V(X.)-V(X  )}  =  E{E{V(X.  )-V(X  )|X  }}  =  Vs(k)Tr, 
l  o  loo  Z-w  k 

keC 

which  proves  the  lemma,  if  condition  (i)  is  true. 

(b)  Let  condition  (ii)  be  satisfied.  Via  proposition  3,  the  series  ^  ]S(k)ir^ 

keC 

converges  absolutely.  Thus  (Chung  (1960),  Th.  2), 


P^im  n-1  ^  S(Xi)  =  ^S(k) \ 


Let  us  now  assume  that  there  exists  state  Z  in  C,  such  that,  P(X  =  Z)  =  1. 

o 

Let  then,  T^  ;  k  _>  0,  denote  the  time  of  the  kth  visit  to  state  Z.  The  sequence, 
{T^},  forms  a  renewal  process,  and  the  process,  {S(X^)},  is  regenerative  with  respect 
to,  {T^}.  But,  due  to  the  ergodicity  assumption,  we  have,  E{T^}  <  °°,  and. 


|s(k) !  =  |E(v(Xn+1)-v(Xn) )xn=k} 1  <  E{|v(xn+1)-v(xn) |/xn=k>  <  B  ;  v  keC 


n+i  n  n 


Therefore, 


(16) 


I ^  s(v  *  - B  *  E*”V  < 


From  Stidham  (1972),  and  due  to  (16)  we  then  conclude, 

n  Tj-1 

p|£im  n"1  ^  S(Xi)  =  E'1  (T1>  *  E  j  £  S(Vj)  = 


From  (15)  and  (17),  we  then  conclude. 


(18) 


Given,  P(X  =  t)  =  1,  let  us  define, 
o 

n 

Yn  =  £  [E(V(X1) |Xi _1>  -  V(Xi)]  ;  n  >  1  (19) 

i=l 

n-1  n-1 

Zn  '  ES<V  '  Z(E{V(Xi+l)  |X1}  '  mt)]  ' 
i=0  i=0 

n 

=  J2  [Etvcx^  Ix^  }  -  v(xi_1)]  - 

i=l 

n 

=  ^2  t(E{V(XJ.)  |X±_1}  -  V(X±))  +  (V(X1)-V(Xi  j^))] 
i=l 

n 

E[V(X.)-V(X,  ,)]  +  Yo  =  V(X)-V(X  )  +  Y  ;  n  >  1 
i  i-1  n  n  o  n  — 

i=l 


Z  =  V(X  )-V(£)  +  Y  ;  n  >  1  (20) 

n  n  n  — 

Due  to  proposition  1,  we  have,  E{|v(X^)|}  <  Vi.  Due  to  the  Markovian 
assumption,  we  have,  E{v(X^) jx^  }  ■  E{V(X^) |x^  Thus,  the  process,  {Y^ } , 

in  (19)  is  a  martingale,  with  respect  to  the  process,  (Xn),  (see  Karlin  et  al  (1975) 


p.  240,  ex.  b) .  We  now  obtain, 


IVl  -  Y„l  -  IE{V(!WIX„>  -  V(Wl  ■  lE(V(X„+l>  -  V<Xn)lX»)  - 

-  -  v(V>l  "  IVi  -  Y„I  i B  +  lv<W  -  v<Vl  - 

*  E<lYn+1  -  *„!«„ . V  +  EtlV(X„+l>  -  V(V  1/  X„.---.X0>  • 

=  B  +  E{ |v(X  l1)  -  V(X  ) l/X  }  <  2  B  ;  V  n  >  1,  due  to  condition  (ii).  (21) 
n+x  n  n  —  — 

Since  now  is  a  Markov  time  with  respect  to  {X^},  and  since  E  {.T^}  <  °°,  we 
obtain  in  conduction  with  (21)  and  corollary  3.1,  page  260,  in  Karlin  et  al  (1975), 

E{Y_  }  =  E{Y.}  =  E{E{V(X  )|X  }  -  V(X  )}  =  0  (22) 

1  1  o  1 

From  (20)  and  (22) ,  we  then  obtain, 

e{zt  }  -  e{v(xt  )  -  v(£)  +  yt  }  «*  vU)  -  vU)  +  e{yt  }  -  0  (23) 

T.-l 

I  1  I 

From  (20)  and  (23),  we  conclude,  Ei  E  S(X  )  I  =  0;  expression  (18)  thus  gives, 

i=0  1 

Es<k>\  ■ 0 

keC 

The  proof  of  the  lemma  is  now  complete. 

Remark  3 

We  note  that  conditions  (i)  and  (ii),  in  lemma  2,  are  such  that  the  one  does 
not  imply  the  other.  Condition  (ii)  does  not  involve  limiting  probabilities;  thus, 
it  may  be  more  applicable  in  practice.  We  note  that  in  the  proof  of  lemma  2,  the 
regenerative  process,  (S(X^)},  may  be  such  that,  S(X^)  -  D,  for  some  D  positive 

and  finite.  Then,  the  proof  works,  via  the  substitution,  S'(X^)  *  S(X^)  +  D.  In 
contrast,  the  regenerative  process,  {S(X.)},  has  been  assumed  nonnegative  in  Stidman 
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Lemma  2  provides  sufficient  conditions  for  the  existence  of  steady-state 
equilibrium,  in  generalized,  irreducible,  aperiodic,  and  ergodic  Markov  chains, 
with  denumerable  state  space. 

4 .  An  Example 

One  of  the  many  applications  of  Markov  chains  lies  with  the  analysis  of  "limited 
feedback  sensing”  random  access  algorithms,  for  computer-communication  data  networks. 
The  "limited  feedback  sensing"  class  of  random  access  algorithms  requires  that  each 
user  monitor  the  feedback  from  the  time  he  generates  a  new  packet,  to  the  time  when 
this  packet  is  successfully  transmitted,  and  it  finds  numerous  applications  in  many 
real  systems.  The  algorithms  within  this  class  frequently  induce  irreducible  and 
aperiodic  Markov  chains,  with  denumerable  state  space.  Sufficient  conditions  for 
the  ergodicity  of  those  chains,  provide  then  lower  bounds  on  the  throughput  of  the 
algorithms.  Sufficient  conditions  for  steady  state  equilibrium,  provide  the  means 
for  the  evaluation  of  useful  algorithmic  statistical  properties.  Here,  we  will 
use  the  results  in  lemma  2  of  this  paper,  to  evaluate  such  properties  for  one  of  the 
algorithms  in  Vvedenskaya  and  Tsybakov  (1982) . 

Let  us  consider  algorithm  A  in  Vvedenskaya  and  Tsybakov.  The  analysis  of  the 

algorithm  is  facilitated  by  the  concept  of  a  marker,  as  described  in  the  above 

reference.  The  marker  can  take  the  integer  values,  -1,  0,  1,  2 .  If  the 

marker  takes  the  value,  -1,  at  some  point  in  time,  it  maintains  this  value,  until  a 

collision  is  encountered.  Upon  the  occurrence  of  the  latter  event,  the  marker  takes 

the  value,  1.  From  that  point  on,  the  marker  updates  its  values,  following  the  rules 

of  the  algorithm,  until  it  takes  again  the  value,  -1,  at  which  point  it  completes  a 

session.  It  then  repeats  the  above  process.  Let  time  be  measured  in  slot  units, 

and  let,  M^,  denote  the  value  of  the  marker  at  time,  i.  If  M^  >  0,  let  0^  denote 

the  number  of  packets  that  at  time  i  are  in  cell  Ifk  of  the  stack,  whose  state 

represents  the  algorithmic  state.  If  M^  *  -1,  let  0*^  denote  the  number  of  packets 

in  cell  #0  of  the  stack.  The  numbers,  {0.*}  and  {M, },  are  random  variables.  Let  us 

’  k  i 


then  define  the  random  vector,  X,  ■  [0  ,  ,  M  ].  From  the  operation  of  the 

i  o  i  1 

algorithm,  it  is  easily  concluded  that  the  process,  {X  };  i  ^  0  is  a  Markov  chain, 

00 

with  state  space,  C  =>  G  ,  where, 

k=-l 

{[0o,61,...,0k,k]  ;  6±  e  Nq};  k>0 
l[Q  ,,-U  ;  9  ,  e  N  };  k  =  -1 

-1  -1  o 

Since  all  the  sets  in  G  above  are  denumerable,  so  is  C.  Furthermore,  the 
process,  {X^};  i  _>  0,  is  irreducible  and  aperiodic. 

Let,  {6^}  and  {m^},  denote  respectively  realizations  of  the  sequences,  {0*} 
and  {M^}.  Given  k£C,  let  us  then  define, 

“k 

v(k)  =  9i  >  0 

i=-l 

It  can  be  easily  verified  that,  E{ |v(Xi+^)-V(X^) |/X^  °  k},  satisfies  condition 
(ii)  in  lemma  2,  and  that, 

e{v(x  - )  -  v(x.)/x,  -  re  ,...,e  ,m]}  «  x  ;  if  e  >2 

l+JL  ixom  o  — 

EtV(Xi+l)  "  V(Xi)/Xi  "  “  X  ;  if  0J  >  2 

E{V(Xi+l)  _  V(V/Xi  =  “  _1  +  X 

E{V(Xi+l)  "  v(V/xi  =  u.-ii)  -  -1  +  X 

E{V(Xi+l)  ‘  v(V/xt  "  1°,  9! . 6m’m))  "  X 

Etv(xi+i)  "  "  [o,-i]>  *  X 

;  where  X  is  the  intensity  of  the  Poisson  user  process. 


Thus,  from  lemma  2  and  the  above,  we  conclude  that  in  the  X  -  region,  where  the 
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process,  {X^};  i  _>  0,  is  ergodic,  we  have, 

^S(k)TTk  -  A  TTk  +  (-1  +  A)  ^  -  0  -*■ 

keC  k:6  and  9  ,}*1  k:6  or  9  “1 

o-l  o-l 

*  £  \  ■ x 

k:  6  or  B  •  1 
o  -1 

The  left  part  of  the  above  equality  represents  the  limiting  probability  of 
successful  transmission.  The  equation  expresses  then  the  fact  that  in  the  X  -  region, 
where  the  process,  {X^};  i  j>  0,  is  ergodic,  the  input  traffic  rate  equals  the  output 
traffic  rate. 

Given  k  in  C,  let  us  now  define, 

V'(k)  -  V,([0o,01 . 0J)  -  >  -1 

For  the  functional,  V'(k),  above,  condition  (li)  in  lemma  2  is  again  satisfied, 

and. 


E{v' (xi+l) 

-  V* 

(xi)/xi  - 

[0  , ...,e  ,m]}  - 

o  m 

i  ; 

if 

0  >  2 
o  — 

E{v* <xi+l) 

-  V’ 

(xi)/x1  - 

te_r-i]> 

2  ; 

if 

0_1  >  2 

E{v'(xi+1) 

-  V' 

(xi)/xi  - 

[1*9^,  .  •  •  i 

,  9  ,m] }- 
n 

-l 

E{v'(xi+1) 

-  V* 

(xi)/xi  = 

[0, 0^ , • • • , 

9  ,m] >= 
m 

-l 

E{v'(xi+1) 

-  V' 

(X^/Xj.  - 

[1,-1]}  - 

E{V,(x1+i>  - 

V' 

(X^/Xj.  -  [0,-1]}  -  0 

Thus,  from  lemma  2  and  the  above,  we  conclude  that  in  the  X  -  region,  where  the 
process,  {X^};  i  _>  0,  is  ergodic,  we  have. 


E  \  +  2  E  E  E  \-° 

keC  k;9  >2,m>0  k:6  >2,m=-l  k:0  =l,m>0  k:0  =0,m>0 

O  -1  o  —  o  — 


k:9  =0  or  0  ,=0 
o  -1 


\  ■ 2-1 b*  E  \  - n] 


k:m=-l 


The  left  part  of  the  above  equation  represents  the  limiting  probability  of  an 

empty  slot.  Also,  ^  ■  E  ^{l},  where  E{l}  is  the  expected  session  length 

k:m=-l 

induced  by  the  algorithm.  Thus,  the  limiting  probability  of  an  empty  slot  can  be 
found  from  the  above  equation,  as  a  function  of  the  expected  session  length. 


References 


Chung,  K.  L.  (1960),  Markov  Chains  with  Stationary  Transition  Probabilities, 
Springer-Verlag,  New  York. 

Kaplan,  M.  (1979),  "A  Sufficient  Condition  for  Nonergodicity  of  a  Markov  Chain," 

IEEE  Trans.  Inf.  Th..  Vol.  IT-25,  No.  4,  July. 

Karlin,  S.  and  Taylor,  H.  M.  (1975),  A  First  Course  in  Stochastic  Processes, 

Academic  Press,  second  edition.  New  York. 

Pakes,  A.  G.  (1969),  "Some  Conditions  for  Ergodicity  of  a  Packet-Switching  Multi- 
Access  Channel,"  Op.  Res..  Vol.  17,  1059-1061. 

Stidham,  S.  (1972),  "Regenerative  Processes  in  the  Theory  of  Queues  with  Applications 
to  the  Alternating-Priority  Queue,"  Adv.  Appl.  Prob.,  4,  542-577. 

Szpankowski,  W.  (1981),  "Ergodicity  Aspects  of  Multidimension  Markov  Chains  with 
Application  to  Communication  Systems  Analysis,"  Technical  Report,  Technical  Univ.  of 
Gdansk,  Institute  of  Telecommunications,  Poland. 

Vvedenskaya,  N.  D.  and  Tsybakov,  B.  S.  (1982),  "Random  Multiple  Access  of  Packets 
to  a  Channel  with  Errors,"  Probl.  Peredachi  Inf..  Vol.  19,  No.  2,  52-68. 


FILMED 

4-85 


DTIC 


